Creation, Analysis and Evaluation of AnnoMI, a Dataset of Expert-Annotated Counselling Dialogues

Research on the analysis of counselling conversations through natural language processing methods has seen remarkable growth in recent years. However, potential this field is still greatly limited by lack access to publicly available therapy dialogues, especially those with expert annotations, but it been alleviated thanks release AnnoMI, first and freely conversation dataset 133 faithfully transcribed expert-annotated demonstrations high- low-quality motivational interviewing (MI)—an effective strategy that evokes client motivation for positive change. In work, we introduce new utterance attributes AnnoMI describe entire data collection process more detail, including dialogue source selection, transcription, annotation, post-processing. Based annotations key MI aspects, carry out thorough analyses respect counselling-related properties utterance, conversation, corpus levels. Furthermore, utterance-level prediction tasks real-world impacts build baseline models. Finally, examine performance models dialogues different topics probe generalisability unseen topics.